Stochastic gradient descent (SGD) and its variants are the main workhorses for solving large-scale optimization problems with nonconvex objective functions. Although the convergence of SGDs in the (strongly) convex case is well-understood, their convergence for nonconvex functions stands on weak mathematical foundations. Most existing studies on the nonconvex convergence of SGD show the complexity results based on either the minimum of the expected gradient norm or the functional sub-optimality gap (for functions with extra structural property) by searching the entire range of iterates. Hence the last iterations of SGDs do not necessarily maintain the same complexity guarantee. This paper shows that an $\epsilon$-stationary point exists in the final iterates of SGDs, given a large enough total iteration budget, $T$, not just anywhere in the entire range of iterates -- a much stronger result than the existing one. Additionally, our analyses allow us to measure the density of the $\epsilon$-stationary points in the final iterates of SGD, and we recover the classical $O(\frac{1}{\sqrt{T}})$ asymptotic rate under various existing assumptions on the objective function and the bounds on the stochastic gradient. As a result of our analyses, we addressed certain myths and legends related to the nonconvex convergence of SGD and posed some thought-provoking questions that could set new directions for research.

Myths, as most readers will know, are stories that explain natural occurrences and express beliefs of right and wrong, while legends are, in the context of this article, popular myths of more recent origin. Myths and legends apply as much to contemporary science as to ancient historical phenomenon. The myths and legends of AI, which examined in this article, will illustrate exemplify that statement. Artificial Intelligence ("AI") is the engineering science of making intelligent machines and software imitate human behaviour and intelligence. The alleged impact of AI has focussed on the behaviour of humans in a world where AI moves centre stage in discharging roles and tasks, which humans have performed historically. Popular publications are the source of many myths and legends which have grown-up around AI Novels from Mary Shelley's Frankenstein, through Samuel Butler's Erewhon to the novel of Brian Aldiss all feature creations which might replace the human being as the dominant specie.

Artificial intelligence, or AI, is the hot and trending topic everyone's been talking about these days. Industries like healthcare, manufacturing, transportation and customer service are already seeing the benefits of embracing this type of technology and what it can do to make better, more efficient processes. By 2020, 85 percent of customer interactions will be managed without a human. As AI starts to touch the industries of recruiting and HR, many experts have a lot to say about it before actively embracing what it can do. But, what is actually true about this type of technology and what is myth?

Artificial intelligence, or AI, is the hot and trending topic everyone's been talking about these days. Industries like healthcare, manufacturing, transportation and customer service are already seeing the benefits of embracing this type of technology and what it can do to make better, more efficient processes. By 2020, 85 percent of customer interactions will be managed without a human. As AI starts to touch the industries of recruiting and HR, many experts have a lot to say about it before actively embracing what it can do. But, what is actually true about this type of technology and what is myth?